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Abstract  -  An  individual  sensor's  information  output  is 
often  insufficient  for  an  application,  with  ambiguities  that 
require  refinement  or  corroboration  by  fusion  with 
information  from  other  sensors.  Fusion  of  multiple 
information  sources  is  performed  to  create  an  information 
product  of  higher  quality  of  information  (Qol)  that  supports 
more  effective  intelligence,  surveillance,  and 
reconnaissance  (ISR).  In  this  paper  we  present  an 
approach  to  determining  the  Qol  attributes  (metadata) 
relevant  to  tracking  and  classification  of  multiple  vehicles, 
and  the  necessary  weighting  (qualifying)  terms,  as 
information  derived  from  multiple  acoustic  sensors  is  fused. 
Field  trial  data  is  used  to  validate  the  conclusions. 
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1  Introduction 

Information  quality  has  been  studied  extensively  in  the 
context  of  data  collected  and  stored  in  data  warehouses  [14, 
15],  and  it  covers  issues  such  as  accuracy,  timeliness, 
completeness,  consistency,  etc.  On  the  other  hand,  quality 
of  information  (Qol)  for  sensor  networks  has  attracted 
significantly  less  attention;  however,  there  is  an  increased 
interest  in  the  area  lately  [16,  17].  Qol  pertains  to  the 
capability  of  assessing  how  well  information  available  to  an 
end-user  (an  analyst,  a  decision  maker,  or  a  computer 
process)  can  be  used  to  reconstruct  portions  of  the  real 
world  that  are  of  interest.  As  stated  metaphorically  in  [18], 
Qol  can  broadly  be  viewed  as  the  ability  of  a  painter  to 
paint,  using  the  information  collected,  desired  aspects  of  the 
real  world  accurately  and  timely  enough  for  viewers  of  the 
painting  to  perform  their  tasks  that  relate  to  the  painted 
world  at  desired  levels  of  effectiveness. 

A  decision-maker  generally  requires  a  well-founded  method 
for  ascribing  confidence  and  importance  to  intelligence 
provided  by  a  wide  spectrum  of  sources.  In  this  paper  we 
consider  sensor  intelligence  (SENSINT)  provided  by  a 
trusted  sensor  network  as  accurately  timed  streams  of 
location  and  classification  information  pertaining  to 
multiple  targets.  The  confidence  estimates  we  require  must 
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come  from  some  combination  of  prior  analysis  of  the 
response  of  the  sensing  service,  projections  of  the  potential 
target  types  and  behaviors  (assuming  a  quiet  environment  in 
this  case),  and  the  measurements  from  the  sensors, 
themselves.  The  novel  analysis  provides  a  method  for 
generating  a  simple,  effective  summary  of  the  Qol  emitted 
by  the  sensor  network.  This  comes  from  using  the 
relationship  between  the  quality  of  the  data  (QoD)  and  the 
resultant  Qol  to  guide  the  derivation  of  simple  Qol 
qualifiers  that  dictate  which  information  sources  should  be 
used  in  a  simple  but  effective  tactical  framework.  The 
experimental  results  comprise  tracking  and  classification  of 
multiple  vehicles  moving  in  inhomogeneous  groups.  We 
discuss  the  potential  benefits  of  derived  Qol  summaries  for 
a  decision-making  task,  and  outline  some  of  the  further 
work  in  which  we  intend  to  explore  them. 

2  Background 

To  generate  comprehensive  situational  awareness  in  a 
location  of  interest,  several  sensors  of  multiple  modalities 
are  deployed.  Each  sensor  provides  partial  information 
about  the  situation;  often  imperfections  such  as  bias, 
envelope  limits,  and  susceptibility  to  environmental  or 
tactical  compromise  impact  the  fundamental  validity  of  that 
information.  Such  imperfect  information  must  be  fused  in  a 
suitable  manner  to  achieve  reasonable  understanding  of  the 
situation.  Fusion  is  intended  both  to  complete  constraint  in 
the  measurement  task,  and  to  improve  the  quality  of 
information  (Qol),  with  the  goal  of  increasing  its  value  (the 
value  of  information,  or  Vol)  to  the  application  or  other 
recipients  [1-3].  The  modeling  performed  as  part  of 
synthesizing  a  fusion  process  ( e.g .  location+Gaussian  noise 
motivates  weighted  averaging)  may  be  rendered  irrelevant 
if  the  source  behaviours  do  not  comply  with  the  modeling 
assumptions.  If  fusion  is  performed  without  understanding 
the  limitations  of  the  sensors  involved,  the  Qol  may,  in  fact, 
deteriorate.  Qol  provides  a  framework  for  reasoning  about 
sensor  network  performance  and  enables  deployment  of 
sensors  for  optimal  utility  [3]. 

The  precise  methods  for  use  of  Qol  estimates  are 
application  specific  [3].  Often,  the  user  has  a  notion  of  what 
is  acceptable  (better  Qol)  and  what  is  unacceptable  (poorer 
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Qol).  Intuitively,  fusion  of  information  is  guided  by  this 
notion  of  good  and  bad,  applying  varying  weight  to  each 
item  of  information  in  the  derivation  of  the  appropriate 
conclusion.  When  it  comes  to  calculation,  the  semantics  of 
the  problem  and  available  assets  enable  us  to  specify  a 
suitable  class  of  fusion  functions  for  using  the  available  Qol 
attributes  (in  operation,  these  are  provided  as  metadata 
accompanying  the  information  [3]),  and  the  particular 
circumstances  give  the  qualifiers  necessary  to  achieving  the 
best  result  in  terms  of  Qol  for  the  particular  instantiation. 

The  meaning  of  an  information  product  is  generally  a 
probabilistic  constraint  on  the  potential  states  of  the  aspects 
of  ground  truth  instrumented  by  the  combined  intelligence 
sources.  For  example,  a  tracking  information  product  will 
state  that  the  locations  of  the  target  that  could  have  given 
rise  to  the  tracker’s  estimate  are  distributed  in  some 
manner,  for  example  as  a  mean  and  a  variance. 

Selection  of  sensor  modalities  also  depends  on  the  notion  of 
good  Qol  desired  for  intelligence,  surveillance,  and 
reconnaissance  (ISR)  [3].  As  a  result,  video  (imaging) 
sensors  are  the  dominant  sensors  deployed  for  gathering 
ISR,  because  the  information  quality  is  regarded  as  obvious 
to  the  user.  We  are  looking  for  an  effective,  pragmatic 
approach  to  fusion  with  the  Qol  in  a  sensor  network  by 
fusing  only  the  sensor  outputs,  which  we  can  quickly  assess 
as  resulting  in  high  Qol.  This  selection  process  includes  an 
example  of  Qol  qualifiers  that  provide  guidance  on  how 
information  and  associated  Qol  should  be  fused. 

In  this  paper  we  explore  the  Qol  in  a  multiple  target 
tracking  scenario.  We  formulate  simple  but  effective 
guidelines  for  basing  direction  of  arrival  (DOA)  angle 
information  of  a  target  from  different  sensors  to  estimate 
the  location  of  the  target.  In  the  signal  processing  literature, 
several  bounds  are  given  for  achievable  location  accuracy. 
For  example,  the  Cramer-Rao  bound  (CRB)  [4,  5]  identifies 
the  overall  accuracy  that  can  be  obtained  for  a  set  of  sensors 
with  identical  noise  statistics.  We  do  not  have  a  simple, 
computationally  frugal  way  of  determining  the  true 
achievable  accuracy  if  the  noise  statistics  change  during  the 
course  of  sensor  usage.  If  we  can  formulate  a  fusion  policy 
that  guarantees  the  true  information  quality  is  sufficiently 
close  to  the  CRB,  then  the  CRB  is  a  candidate  Qol 
approximator  [5].  Intuitively,  the  best  quality  may  be 
achieved  if  the  sensors  with  minimal  noise  characteristics 
are  used.  A  policy  based  on  this  intuition  dictates  the  usage 
of  sensors  with  high  signal  to  noise  ratio  at  any  given  time. 

In  the  case  of  vehicle  tracking  using  a  Kalman  filter,  we  do 
not  have  a  simple  CRB.  When  we  address  a  joint  purpose  of 
tracking  and  classification,  the  accuracy  components  of  Qol 
are  estimated  probability  distributions  of  location  and 
vehicle  type. 

Rather  than  presenting  a  decision-maker  with  panoply  of 
information  quality  indicators,  he/she  may  be  better  served 
with  one  overall  Qol  indicator  for  each  target,  tailored  to 
his/her  specific  decision  task.  This  will  require  an  ante  hoc 
analysis  of  the  range  of  situations  within  which  decisions 


are  to  be  made.  This  means  that  the  information  quality 
indicator  is  conditional  on  some  description  of  the 
circumstances.  Intuitively,  the  more  detailed  this 
description,  the  more  effective  we  would  expect  a  quality 
indicator  to  be.  We  will  analyze  this  presumption  in  a 
follow-on  paper.  For  now,  we  simply  demonstrate  that  it  is 
possible  to  summarize  the  information  quality  from  a 
sensing  service  to  give  an  indicator  function  that  trends 
positively  with  a  correct  decision. 

This  paper  suggests  an  estimate  of  a  suitable  overall  Qol 
value  from  individual  Qol  values  generated  by  the  various 
intelligence  sources — in  our  case  the  angle  estimator, 
tracker,  classifier,  etc.  To  aid  in  the  derivation  and 
exploitation  of  fusion  for  Qol,  we  use  the  following 
problem  statement: 

Problem  statement:  Identify  any  heavy  tracked  vehicle 
approaching  the  northbound  gate  and  intercept  it.  The 

sensor  suites  available  to  track  and  identify  the  vehicles  are 
acoustic  sensors. 

In  section  3  we  present  the  architecture  for  the  tracking  and 
classification  of  multiple  targets  using  sensor  fusion  for 
Qol.  In  section  4  we  present  the  Qol-based  fusion  to 
achieve  better  quality  of  tracking  and  classification  of 
multiple  vehicles  using  acoustic  sensor  arrays.  We  also 
discuss  issues  involved  in  achieving  better  Qol.  In  section  5 
we  present  the  discussions  and  further  work.  Section  6 
presents  conclusions. 

3  Framework  for  simple  Qol  guided 
sensor  fusion 

There  are  six  acoustic  sensor  arrays  deployed  in  the  middle 
of  the  track,  as  shown  in  figure  1.  Each  acoustic  sensor 
array  consists  of  seven  microphones  with  six  of  the 
microphones  placed  on  the  vertices  of  a  regular  hexagon 
(circular  array),  and  the  seventh  microphone  placed  at  the 
center  of  the  array.  Each  array  is  capable  of  determining  the 
direction  of  arrival  (DOA)  of  an  acoustic  signal  that  may  be 
emanating  from  a  vehicle  using  hyper-spectral  techniques, 
such  as  minimum  variance  distortionless  response  (MVDR) 
or  multiple  signal  classification  (MUSIC)  algorithms  [6]. 
The  target  set  consists  of  one  heavy  tracked  vehicle,  one 
light  tracked  vehicle,  one  heavy  wheeled  truck,  and  one 
light  wheeled  truck.  Initially,  all  the  vehicles  are  stationed 
on  the  north  end  of  the  track  and  idling.  A  test-run  consists 
of  the  heavy  tracked  vehicle  followed  by  the  heavy  wheeled 
truck,  the  light  wheeled  truck,  and  then  the  light  tracked 
vehicle,  traveling  at  16  to  32  Km/h  with  a  separation  of 
100-200m  between  them.  Acoustic  signals  from  the 
vehicles  are  analyzed  to  determine  the  DOAs  and  their 
classification  at 
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Figure  1.  Tracking  scenario  with  acoustic  sensors 

each  sensor  array.  Figure  2  shows  the  overall  scheme  for 
processing  the  information  to  determine  the  individual 
tracks  and  classification  of  the  vehicles. 


Figure  2:  Architecture  for  Tracking  and  Classification  of 
multiple  vehicles 

There  are  three  major  modules  in  the  architecture  for  the 
vehicle  tracking  application,  namely,  (a)  Estimation  of 
target  coordinates  (Target  tracks)  using  DO  As  of  targets 
from  different  sensor  arrays,  (b)  Kalman  filtering,  and  (c) 
classifier.  In  figure  2,  the  “DOA  Estimation”  module 
generates  six  (number  of  mics  in  the  array  minus  one)  DOA 
angles  per  sensor  array.  The  DOA  angle  corresponding  to 
the  strongest  signal  is  listed  first,  followed  by  the  others 
according  to  their  signal  strengths.  Notice  that  there  are 
only 
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Figure  3:  Output  of  a  sensor  array  showing  four  angle 
tracks  corresponding  to  four  vehicles 


four  vehicles;  despite  this,  each  sensor  array  generates  six 
DOA  angles,  two  of  which  are  artifacts.  It  is  possible  that 
some  of  the  artifacts  are  due  to  the  signals  reflected  by  a 
nearby  building  or  an  unknown  acoustic  signal  source  in  the 
vicinity.  A  typical  output  of  DOAs  generated  by  a  sensor 
array  is  shown  in  figure  3.  One  can  clearly  see  four 
different  angle  tracks.  As  the  target  comes  close  to  the 
closest  point  of  approach  (CPA)  of  a  sensor  array,  the 
target’s  signal  becomes  the  strongest  and  its  DOA  will  be 
listed  first  irrespective  of  its  order  among  the  vehicles, 
which  can  be  seen  clearly  in  figure  3.  Each  sensor  array 
outputs  its  metadata  information  as  follows: 


Time 

Sensor 

DOA1 

CLS1 

SNR1 

DOA2 

coords 

Table  1:  Metadata  from  a  sensor  array 


The  metadata  consists  of  time-stamp,  sensor  coordinates, 
DOA1  corresponding  to  the  strongest  signal,  its 
classification,  and  its  SNR,  followed  by  the  DOA2 
corresponding  to  the  2nd  strongest  signal,  its  classification, 
its  SNR,  etc.  The  DOAs  from  the  sensor  arrays  are  used  to 
generate  the  target  coordinates  and  tracks  in  the  module 
“Target  Tracks”  in  figure  2.  Initially,  there  will  be  several 
false  tracks  that  will  die  out  shortly  after  the  targets  move. 
Bank  of  Kalman  filters  [6]  are  used  to  track  the  target 
tracks.  The  predicted  coordinates  of  each  target  are  used  to 
generate  the  expected  DOAs  of  each  target,  which  are  used 
for  track  association,  as  shown  in  figure  2.  Note  that  track 
association  will  not  happen  until  tracks  are  formed  clearly. 
Classifier  output  is  also  used  to  assist  the  target  tracks. 

4  Qol  based  fusion  of  DOAs  to 
estimate  target  location 

If  there  are  only  two  sensor  arrays,  then  the  only  option  is  to 
use  both  the  DOAs  from  both  the  sensor  arrays  to  estimate 
the  position  [7,  8]  of  the  target.  However,  if  multiple  arrays 
are  present,  then  there  are  several  ways  one  can  determine 
the  target  location.  Figure  4  shows  the  geometry  for 
localization  using  DOAs.  To  make  the  presentation  suitably 


Figure  4:  Geometry  for  Localization 

self-contained,  localization  formulation  [8]  is  presented 
here.  Let  p.  represent  the  location  of  ith  sensor  array,  while 

Ui  and  Ui  represent  the  unit  vectors  in  the  direction  of  DOA 
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and  its  normal,  respectively,  and  P0is  the  location  of  the 


target.  Then  from  the  geometry 

[9], 

'u,p; 

U2 

Po  = 

U2P2 

un 

UnPn 

a  linear  equation  (1)  can  be  solved  for  target  position  P0 
using  least  squares  methods.  If  the  signal  to  noise  ratio 
(SNR)  at  all  sensor  arrays  is  more  or  less  the  same,  then  use 
of  all  DO  As  from  all  the  sensors  in  (1)  would  result  in  best 
estimate  of  the  target  location.  CRB  gives  the  error  limit  on 
the  localization  based  on  the  SNR  [4,  5].  This  bound  could 
be  used  to  measure  the  quality  of  location  estimation 
(indicator  of  Qol)  on  the  localization.  If  the  SNR  at  some  of 
the  sensor  arrays  is  widely  different,  then  use  of  all  DOAs 
in  (1)  would  result  in  poor  estimation  of  target  location.  To 
estimate  the  SNR  at  each  sensor  array,  let  us  consider  the 
propagation  loss  in  acoustic  signal  ‘S'  that  is  inversely 
proportionate  to  the  square  of  distance 


s=%>  ® 
where  4 A ’  is  the  signal  strength  at  the  source  and  4 d ’ 
denotes  the  distance  between  the  source  and  the  sensor.  For 
the  track  shown  in  figure  5,  the  ratio  between  the  closest 
and  the  farthest  distance  for  a  sensor  array  is  approximately 
15,  which  results  in  a  difference  of  23  db  in  SNR  between 
the  sensor  array  that  is  closest  to  the  target  and  the  sensor 
array  that  is  farthest.  This  results  in  DOA  accuracy  from  2° 
to  10°.  Realistic  estimation  of  the  CRB  and,  hence,  the 
quality  of  target  location  estimation,  is  not  possible.  So, 
blind  use  of  equation  (1)  does  not  provide  a  meaningful 
estimate  of  target  location  (that  is,  good  Qol),  as  seen  in 
figure  5. 


Estimation  of  Target  Position 


Notice  that  when  the  sensor  arrays  and  the  targets  are  almost 
in  line,  even  small  errors  in  DOAs  result  in  large  errors. 
When  the  target  is  close  to  the  mid-range,  all  sensor  arrays 
provide  a  reasonable  estimate  of  DOAs,  resulting  in 
estimates  that  are  close  to  the  track — since  the  arrays  farther 
from  the  target  have  more  error  than  the  ones  close  to  the 
target,  the  error  in  estimation  of  the  target  location  changes 
from  large  to  small,  resulting  in  loops  like  the  one  in  figure 


5.  In  figure  6,  the  target  locations  are  estimated  using  only 
the  DOAs,  which  are  close  to  the  predicted  DOAs  of  the 
expected  target  location.  This  process  of  selected  fusion  of 
DOAs  resulted  in  better  Qol. 

Measure  of  Qol  for  Target  Location  Estimation:  At  each 
stage  in  figure  2,  one  can  assign  a  value  to  the  quality.  The 
quality  of  coordinate  estimation  depends  only  on  the  DOA 
estimation.  If  these  DOAs  deviate  from  the  actual  values, 
the  estimated  target  coordinates  will  be  in  significant  error, 
resulting  in  poor  quality.  Hence,  we  propose  the  Qol  to  be 
inversely  proportional  to  the  difference  between  the 
estimated  DOA,  6le ,  and  the  actual  DOAs,  6a ,  for  each 
sensor  array  4f  and  it  is  given. 


a=I,- 
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^  w-ew 


(3) 


Qol  based  estimation  of  Target  Position 


Figure  6:  Selective  DOA  fusion  for  Target  Location 


The  proposed  measure  for  Qol  gives  us  high  confidence  if  it 
has  large  value,  that  is,  the  estimated  values  are  close  to  the 
actual  values.  In  reality  the  actual  values  may  not  be 
available  or  predicted  apriori.  In  such  a  case,  one  can  use  the 
difference  between  the  measured  values  and  the  predicted 
values  using  the  predicted  coordinates  by  the  Kalman  filter. 
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Figure  7:  (a)  Estimation  accuracy  (b)  Qol  measure 

In  figure  7,  we  presented  the  distance  between  the  actual 
target  location  and  the  estimated  position.  Figure  7  also 
shows  the  quality  of  the  target  estimation  by  (3)  with  the 
upper  limit  set  at  10.  Clearly,  from  figure  7,  we  note  that  the 
target  coordinates  are  close  to  the  ground  truth  when  Qol  is 
high. 
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4.1  Tracking  by  Kalman  Filters 

The  estimated  target  positions  are  fed  to  multiple  Kalman 
filters  [6]  for  tracking  purposes.  Kalman  filters  estimate  the 
dynamic  model  of  each  target  and  update  the  track 
coordinates  using  both  the  model  and  the  input.  One  of  the 
parameters  it  estimates  is  the  dynamic  model  of  the  track, 
which  includes  the  covariance  of  the  data.  If  the  diagonal 
coefficients  of  the  covariance  matrix  vary  by  a  large 
number,  it  implies  that  the  track  coordinates  are  changing 
widely,  indicating  that  the  quality  of  information  (quality  of 
tracking)  is  not  very  good.  If  the  coefficient  is  small,  the 
track  is  following  the  model  closely  and  the  Qol  is  good. 
Figure  8  shows  the  output  of  the  Kalman  filters  for  targets  1 
to  4. 


Target  #1  Target  #2 


Target  #3  Target  #4 


Easting  (m)  Easting  (m) 

Figure  8:  Outputs  of  Kalman  filters  for  all  four  targets 

Measure  of  Qol  for  Kalman  Filtered  Output:  Since  the 
coefficients  provide  us  a  measure  on  the  variation  of  the 
coordinates,  we  propose  the  Qol  of  Kalman  filtered  output 
as 


where  Ct  i  is  the  (/,  /)  coefficient  of  the  covariance  matrix 

for  the  Kalman  filter  tracking  a  particular  target.  Figure  9 
shows  how  the  Qol  defined  by  (4)  relates  to  the  accuracy  of 
tracking. 


4.2  Vehicle  Classification  and  its  Qol 

In  order  to  aid  track  association,  a  multivariate  Gaussian 
(MVG)  classifier  [9-12]  is  developed  for  vehicles.  The 
acoustic  signal  of  a  ground  vehicle,  s(t),  can  be  modeled  at 

any  time,  t,  by  a  coupled  harmonic  signal  model  as 

N 

«(o = X  Ak  c°s(27!:k  u + <Pk ) + »(o  (5) 

k= 1 

where  k  is  the  harmonic  number,  Ak  is  the  amplitude  of  the 
kth  harmonic,  /0  is  the  fundamental  frequency,  (j)k  is  the 
phase  of  the  kth  harmonic,  N  is  the  total  number  of 
harmonics,  and  n(t)  is  the  additive  white  Gaussian  noise. 
The  fundamental  frequency  is  usually  related  to  the  RPM  of 
the  motor  and  determined  by  the  cylinder-firing  rate.  The 
number  of  (detectable)  harmonics,  N ,  is  a  function  of  the 
vehicle  type,  detection  range,  and  the  classification 
algorithm  used  [11-13].  Figure  10  shows  the  spectrogram  of 
target  #1.  The  harmonics  are  clearly  visible  in  figure  10. 
Notice  also  that  the  frequency  changes  as  the  Doppler 
changes. 
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Figure  9:  Coefficients  of  Covariance  Matrix 


Spectrogram  of  target  #1 


Figure  10:  Spectrogram  of  target  #1  signals 

•  MVG  Classifier 

Let  us  assume  that  we  need  to  classify  a  pattern 
X  =  (xj  x2  •  •  • ,  xN )  into  one  of  the  R  categories,  and  each 
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pattern  is  governed  by  one  of  the  R  distinct  probability 
density  functions  p(X\j),  y  =  1, ••*,/? ,  where  p(X\j)  is 
the  probability  of  occurrence  of  pattern  X ,  given  it  belongs 
to  category  y.  Now,  in  our  case,  X  =  [xxx2  ••*,x7V]ris  a 

vector  of  N  harmonic  magnitudes,  where  T  denotes  the 
transpose.  Assuming  they  obey  the  normal  distribution,  then 
the  multivariate  normal  probability  distribution  of  the 
pattern  X  is  given  by 

P(X)  °  (2*r  izr  exp{~K*-y-")r  £"<*-«>}  <«> 


where  the  mean,  M  and  the  covariance  matrix  X  are  defined 
as  M  =  E{X }  =  [ml.m2,---,mN]T 


X  =  E{(X-  M)(X  -  Mf }  = 


aN,  1 


(7) 


and  =E\pp-mpJxq-mq)\,p,q  =  \,2,-",N.  It  is 
assumed  that  the  a  priori  probability  y>(y)  and  the  N- 
variate  normal  probability  function  p{X  |y)  are  known  for 
each  y.  That  is,  we  know  R  normal  density  functions.  Let  us 
denote  the  mean  vectors  Mt  and  the  covariance  matrices  Z . 
for  j  m  1,  •  •  • ,  then  we  can  write 


p(X\H,)  =  - 


(2  ri) 


N/2 


il/2 


■ex  — 


(X-Mf  YThX-M,)} 


(8) 

for  all  j  e  {l,  2,  •  •  • ,  7?}  where  M.  =  ( mjX , mj2 ,  •  •  • ,  mjN )  .  We 

used  the  Matlab  function  “classify”  to  classify  the  targets, 
and  the  classification  results  of  signals  from  the  direction  of 
target  #1  are  shown  in  figure  11.  While  the  classifier  is 
unable  to  classify  the  target  correctly  most  of  the  time, 
hence  it  is  not  good  enough  for  track  association.  The 
reasons  for  misclassification  could  be:  (a)  varying  speeds  of 
the  vehicle  different  from  the  training  data  set,  (b)  shift  in 
gears  can  cause  abrupt  changes  in  the  engine  RPM,  and  (c) 
change  in  the  road  conditions  compared  to  the  training  data. 
The  problem  is  further  complicated  by  the  presence  of  other 
vehicles.  We  find  tracking  a  dominant  frequency  from  each 
vehicle  and  its  SNR  provides  a  better  Qol  on  the  vehicle 
classification. 
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Figure  11:  Classification  of  signals  from  target  #1  direction 


and  approaches  another,  the  harmonic  frequency  is  tracked 
on  the  sensor  array  that  is  closest  to  the  target.  The  fusion  of 
the  data  from  different  arrays  is  the  key  for  prolonged 
tracking  and  accurate  classification. 
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Figure  12:  Classification  of  signals  from  direction  of  targets 
1  &  2  using  dominant  frequency  tracking 


Distance  from  the  Actual  Location 


Fusion  for  Qol:  In  order  to  isolate  the  signals  from 
individual  targets,  we  perform  spatially  filtering  of  the 
signals  based  on  the  bearing  estimates  of  each  target.  We 
extract  and  record  the  dominant  harmonic  and  its  amplitude 
for  each  target.  We  track  this  dominant  harmonic  frequency 
for  each  target  and  classify  the  target  based  on  this  single 
dominant  harmonic.  As  the  target  leaves  one  sensor  array 


Figure  13:  Qol  of  classifier 

The  resulting  classifications  of  two  targets  are  given  in 
figure  12.  As  in  figure  11,  the  different  target  classifications 
are  depicted  using  different  colors:  black  for  target  #1,  cyan 
for  target  #2,  green  for  target  #3,  and  pink  for  target  #4. 
Clearly,  tracking  dominant  harmonics  of  each  target 
provided  a  better  classification  and  can  be  used  in  the  track 
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association,  as  shown  in  figure  2.  From  figure  12,  one 
notices  that  for  the  same  track,  occasionally  target  #1  is 
classified  as  something  else.  This  can  be  easily  corrected 
based  on  the  positions  of  each  target  and  their  continued 
classification.  Figure  13  shows  how  the  Qol  defined  by  (9) 
relates  to  the  accuracy  of  tracking. 

Measure  of  Qol  for  Classifier  Output:  Clearly,  the 
dominant  harmonic  frequency  of  individual  target  is  used 
for  its  classification — the  natural  choice  for  Qol  of  the 
classification  would  be  its  amplitude  or  its  SNR.  The  Qol  of 
the  classifier  is  defined  as 

Qc=S-th  (9) 

where  ‘S'  denotes  the  SNR  of  the  dominant  harmonic 
frequency  for  the  target  in  question,  and  th  is  the  threshold. 

4.3  Overall  Qol 

In  the  tracking  approach  illustrated  in  figure  2,  three 
different  key  parts  are  identified — namely,  location 
estimator,  Kalman  filter  tracking,  and  target  classifier.  For 
each  one  of  these  key  parts  a  Qol  measure  is  defined  in 
equation  (3),  (4)  and  (9).  These  Qol  measurements  are 
intended  to  provide  an  appropriate  level  of  confidence  to 
the  observer  as  to  how  well  each  part  is  working.  An 
effective  overall  Qol  estimate  for  the  entire  tracking  system 
would  be  highly  valuable  and  would  have  to  be  generated 
using  the  individual  Qols. 

A  simple,  conservative/pessimistic  use  of  the  three  location 
qualities  would  adopt  the  minimum  quality.  However,  since 
each  is  an  approximation,  we  intuitively  recognize  that  this 
will  generally  be  pessimistic,  and  so  actual  opportunities 
(that  would  have  been  recognized  from  a  god’s-eye  view) 
will  be  missed.  For  this  reason,  we  explore  a  simple  Qol 
re-estimator  that  emits  a  linear  combination  of  the  three. 
We  seek  to  increase  the  effectiveness  of  the  information 
source  by  producing  a  good  Qol  estimator  for  a  specific 
decision-making  scenario.  Success  will  be  indicated  by  a 
statistically  significant  benefit  to  using  a  Qol  combinator 
over  using  the  worst.  Of  course,  we  can  design  an 
experiment  that  favors  a  combinator.  In  fact,  we  do  so,  but 
note  that  the  experiment  is  realistic. 

As  well  as  Qol  attributes,  themselves — falling  generally 
under  accuracy,  timeliness,  and  trust  types — we  need  Qol 
qualifiers  to  inform  us  as  to  how  those  attributes  should  be 
put  to  use.  We  are  exploring  the  space  of  such  qualifiers, 
but  for  our  purposes  here,  we  define  two  general,  simple 
regimes.  The  first,  in  which  the  worst  Qol  is  taken,  is  a 
pessimistic  bound  estimator,  in  which  the  qualifier  is  that 
we  pass  on  the  minimum  or  worst  quality.  The  second  is  a 
weighted  sum,  in  which  each  information  source  is  assigned 
a  weight.  This  weight  is,  in  fact,  a  function  of  the  data 
quality,  and  its  shape  is  seen  in  our  results.  In  further  work 
we  hope  to  derive  this  function  from  first  principles.  We 
define  a  straw-doll  overall  Qol  function  as  a  weighted  sum 
of  individual  Qols: 


Q  =  w,Q,  +  wkQk  +  wcQc  ( 1 0) 

Individual  weights  are  to  be  either  synthesized,  based  on  the 
role  each  part  plays  in  the  overall  quality  of  the  system,  or 
learned  through  experience.  Some  weights  may  have  high 
value  compared  to  others  if  the  quality  of  the  overall  system 
is  more  dependent  on  those  parts.  We  have  a  choice  as  to 
which  attributes  to  aggregate,  and  which  to  leave  separate. 

In  the  introductory  section  we  suggested  a  decision-making 
task  of  intercepting  any  tracked  vehicle  that  is  traveling 
northbound.  This  requires  tracking  the  vehicle  accurately 
and  classifying  it  as  the  tracked  vehicle.  For  example,  if  the 
classifier  can  classify  the  vehicle  as  a  tracked  vehicle  with 
high  confidence — that  is,  with  high  Qol  value — the 
battlefield  commander  can  order  the  tracked  vehicle  to  be 
intercepted  when  it  is  approaching.  The  target  tracker  Qol 
along  with  the  Kalman  filter  Qol  show  how  well  the  target 
is  being  tracked,  and  the  classifier  identifies  it  as  a  tracked 
vehicle.  Figure  14  shows  the  distance  between  the  check 
post  located  at  (160,  700)  and  tracked  vehicle,  individual 
Qols,  and  the  overall  Qol.  For  overall  Qol,  we  used  the 
mean  of  all  Qols  ( vy.  =  1/3 )  for  all  in  (10).  These  weights 
can  be  appropriately  learned  with  experience. 
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Figure  14:  Distance  from  check  post  and  Qols 

The  decision  to  intercept  the  vehicle  can  be  made  during  the 
northbound  track,  while  the  overall  Qol  is  good — that  is, 
above  a  threshold  of  5. 

5  Discussion  and  further  work 

Development  of  effective  estimation  and  communication  of 
information  quality  to  decision-makers  is  an  essential 
research  target,  when  the  range  and  number  of  sensing 
assets  employed  in  sensing  mission  is  rising  rapidly.  There 
are  two  ends  of  the  spectrum:  theoretical  ab  initio  synthesis, 
and  empirical  discovery.  Here  we  have  begun  to  explore 
empirical  approaches,  with  an  eye  to  recognizing  the 
theoretical  issues  as  they  become  clear  during  testing.  We 
have  produced  some  basic,  intuitive  indicators  of  Qol 
calculated  directly  on  the  output  of  the  sensing  system  and 
compared  them  with  a  view  of  ground  truth  known  from  the 
experiment  that  would  not  be  available  in  normal  operation. 
We  find  that  these  correlate  well,  particularly  when 
combined  in  a  simple  but  effective  manner.  This  suggests 
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that  some  investment  in  analysis  of  existing  test  data  against 
realistic  decision  making  requirements  is  warranted,  and 
that  the  use  of  simple  combinators  may  be  effective  where 
extensive  calculation  in  theater  is  not  practical. 

In  the  Kalman  filter,  we  find  in  common  with  other 
researchers  that  the  covariance  matrix  gives  us  a  good 
handle  on  the  accuracy  of  the  filter’s  predictions,  and,  for 
our  purposes,  the  quality  of  the  information.  The  errors  in 
DOA  estimates  are  well-behaved,  so  we  will  filter  the  DOA 
estimates  themselves,  separately,  or  in  combination  with  the 
derived  Cartesian  coordinates.  The  classifier  success  rate 
derives  from  processing  excerpts  from  the  same  data  as 
used  in  location  estimation.  The  quality  of  data  (QoD)  will, 
therefore,  impact  both.  This  leads  us  to  expect  the  observed 
correlation  between  tracking  fidelity  and  classification 
success  rates. 

6  Conclusions 

Effective  construction  and  maintenance  of  situational 
awareness  during  tactical  activity  generally  requires  fusion 
of  information  sources.  Estimates  of  the  accuracy, 
confidence,  and  believability  of  information  products  are 
essential  in  the  decision-making  process.  Bandwidth  and 
electrical  power  constrained  scenarios  strongly  motivate  the 
minimization  of  computation  and  communications  required. 
In  this  paper,  tracking  and  classification  accuracy  of  Qol 
attribute  values  are  estimated,  and  candidate  Qol  qualifiers 
are  defined  for  a  tracking  scenario.  These  are  simple  yes/no 
qualifiers  that  dictate  which  information  should  be  fused, 
but  to  which  simple  policy  gives  an  effective  method  with 
low  computational  complexity  and  the  opportunity  to  report 
summarized  Qol  to  a  user  in  a  quickly  digestible  form  to 
simplify  decision-making  under  high  loading. 
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